Have I seen you before? Principles of Bayesian predictive classification revisited

A general inductive Bayesian classification framework is introduced for data from multiple finite alphabets using predictive representations based on random urn models and generalized exchangeability. We develop a novel principle of generative supervised and semi-supervised probabilistic classification based on marginalizing simultaneous predictive classification probabilities for all test items jointly given training data. Both the simultaneous and marginalized classifiers have attractive theoretical properties and are illustrated in numerical examples with sparse training data to achieve higher correct classification rates compared to a standard predictive classifier which assigns all test items independently to a source. Optimal simultaneous and marginal supervised predictive classifiers are shown to become equivalent classification rules under generalized exchangeability when the amount of training data increases. The predictive framework can under the form of exchangeability considered here coherently handle missing observations without added computational complexity. We discuss briefly a range of options for generalizing the simultaneous classification framework to allow for different data types and less restrictive modeling assumptions.